35 research outputs found
Sharp Bounds for Optimal Decoding of Low Density Parity Check Codes
Consider communication over a binary-input memoryless output-symmetric
channel with low density parity check (LDPC) codes and maximum a posteriori
(MAP) decoding. The replica method of spin glass theory allows to conjecture an
analytic formula for the average input-output conditional entropy per bit in
the infinite block length limit. Montanari proved a lower bound for this
entropy, in the case of LDPC ensembles with convex check degree polynomial,
which matches the replica formula. Here we extend this lower bound to any
irregular LDPC ensemble. The new feature of our work is an analysis of the
second derivative of the conditional input-output entropy with respect to
noise. A close relation arises between this second derivative and correlation
or mutual information of codebits. This allows us to extend the realm of the
interpolation method, in particular we show how channel symmetry allows to
control the fluctuations of the overlap parameters.Comment: 40 Pages, Submitted to IEEE Transactions on Information Theor
The Effect of Saturation on Belief Propagation Decoding of LDPC Codes
We consider the effect of LLR saturation on belief propagation decoding of
low-density parity-check codes. Saturation occurs universally in practice and
is known to have a significant effect on error floor performance. Our focus is
on threshold analysis and stability of density evolution.
We analyze the decoder for certain low-density parity-check code ensembles
and show that belief propagation decoding generally degrades gracefully with
saturation. Stability of density evolution is, on the other hand, rather
strongly affected by saturation and the asymptotic qualitative effect of
saturation is similar to reduction of variable node degree by one.Comment: Submitted to ISIT. Longer version to be submitted to IT Transactions
in preparatio
Decay of Correlations for Sparse Graph Error Correcting Codes
The subject of this paper is transmission over a general class of
binary-input memoryless symmetric channels using error correcting codes based
on sparse graphs, namely low-density generator-matrix and low-density
parity-check codes. The optimal (or ideal) decoder based on the posterior
measure over the code bits, and its relationship to the sub-optimal belief
propagation decoder, are investigated. We consider the correlation (or
covariance) between two codebits, averaged over the noise realizations, as a
function of the graph distance, for the optimal decoder. Our main result is
that this correlation decays exponentially fast for fixed general low-density
generator-matrix codes and high enough noise parameter, and also for fixed
general low-density parity-check codes and low enough noise parameter. This has
many consequences. Appropriate performance curves - called GEXIT functions - of
the belief propagation and optimal decoders match in high/low noise regimes.
This means that in high/low noise regimes the performance curves of the optimal
decoder can be computed by density evolution. Another interpretation is that
the replica predictions of spin-glass theory are exact. Our methods are rather
general and use cluster expansions first developed in the context of
mathematical statistical mechanics.Comment: 40 pages, Submitted to SIAM Journal of Discrete Mathematic
Statistical physics methods for sparse graph codes
This thesis deals with the asymptotic analysis of coding systems based on sparse graph codes. The goal of this work is to analyze the decoder performance when transmitting over a general binary-input memoryless symmetric-output (BMS) channel. We consider the two most fundamental decoders, the optimal maximum a posteriori (MAP) decoder and the sub-optimal belief propagation (BP) decoder. The BP decoder has low-complexity and its performance analysis is, hence, of great interest. The MAP decoder, on the other hand, is computationally expensive. However, the MAP decoder analysis provides fundamental limits on the code performance. As a result, the MAP-decoding analysis is important in designing codes which achieve the ultimate Shannon limit. It would be fair to say that, over the binary erasure channel (BEC), the performance of the MAP and BP decoder has been thoroughly understood. However, much less is known in the case of transmission over general BMS channels. The combinatorial methods used for analyzing the case of BEC do not extend easily to the general case. The main goal of this thesis is to advance the analysis in the case of transmission over general BMS channels. To do this, we use the recent convergence of statistical physics and coding theory. Sparse graph codes can be mapped into appropriate statistical physics spin-glass models. This allows us to use sophisticated methods from rigorous statistical mechanics like the correlation inequalities, interpolation method and cluster expansions for the purpose of our analysis. One of the main results of this thesis is that in some regimes of noise, the BP decoder is optimal for a typical code in an ensemble of codes. This result is a pleasing extension of the same result for the case of BEC. An important consequence of our results is that the heuristic predictions of the replica and cavity methods of spin-glass theory are correct in the realm of sparse graph codes